Biometric signal analysis for communication enhancement and transformation

ABSTRACT

Techniques are described for data transformation performed based on a current emotional state of the user who provided input data, the emotional state determined based on biometric data for the user. Sensor(s) may generate biometric data that indicates physiological characteristic(s) of the user, and an emotional state of the user is determined based on the biometric data. Different dictionaries and/or dictionary entries may be used in translation, depending on the emotional state of the sender when the data was input. In some implementations, the emotional state of the sending user may be used to infer or otherwise determine that a translation was incorrect. The input data may be transformed to include information indicating the current emotional state of the sending user when they provided the input data. For example, the output text may be presented in a user interface with an icon and/or other indication of the sender&#39;s emotional state.

CROSS-REFERENCE TO RELATED APPLICATION

The present disclosure is related to, and claims priority to, U.S.Provisional Patent Application Ser. No. 62/381,170, titled “BiometricSignal Analysis for Data Transformation,” which was filed on Aug. 30,2016, the entirety of which is incorporated by reference into thepresent disclosure.

BACKGROUND

Translating from one natural language to another is a challengingproblem to solve using computer software, given the complexity that isinherent in natural languages that have evolved over millennia throughuse in different societies. In general, natural language processing(NLP) techniques have been developed to analyze spoken or written inputand translate the input to generate output in a different designatednatural language. Existing, traditional translation solutions maytranslate each word or phrase individually, and may therefore introducetranslation errors through lack of contextual awareness. Traditionaltranslation solutions may exhibit translation errors in instances wherewords have a similar spelling and/or sound (e.g., “be” vs. “bee”, “dye”vs. “die”, etc.). Also, traditional translation solutions may alsogenerate translated sentences in an order that is different than theorder that would normally be used in the output language. Further,traditional transcription solutions that are intended to transcribespeech to text may also exhibit errors due to lack of contextualawareness, similar sounding words (e.g., homophones, homonyms) in theinput, and/or other causes.

SUMMARY

Implementations of the present disclosure are generally directed toenhancing and/or transforming data that is transmitted incommunications. More specifically, implementations are directed toenhancing, translating, and/or transforming data that is communicatedbetween users, based at least partly on an emotional state of a sendinguser who provides the data to be communicated to a receiving user, theemotional state determined based at least partly on biometric datadescribing a current physiological state of the sending user.

In general, innovative aspects of the subject matter described in thisspecification can be embodied in methods that include actions of:receiving input data that is provided, by a first user, forcommunication to a second user; determining a state of the first userduring a time period when the input data is provided, the statedetermined based on biometric data that is generated by at least onesensor device measuring at least one physiological characteristic of thefirst user during the time period; transforming the input data togenerate output data, the transforming based at least partly on thestate; and communicating the output data to be presented to the seconduser through a computing device.

Implementations can optionally include one or more of the followingfeatures: transforming the input data includes translating the inputdata from a first natural language to generate the output data in asecond natural language; the translating employs at least one dictionarythat provides a mapping from at least a portion of the input data to atleast a portion of the output data, the mapping corresponding to thestate; the translating includes performing a first translation of atleast a portion of the input data from the first natural language togenerate at least a portion of the output data in the second naturallanguage, determining, based on the state, that the first translation isincorrect, and performing a second translation of at least the portionof the input data from the first natural language to generate at leastthe portion of the output data in the second natural language;communicating the output data causes an indicator to be presented to thesecond user, the indicator corresponding to the state of the first user;the output data is generated to include the indicator to be presented tothe second user; the output data is generated to include metadata thatcorresponds to at least one of the indicator or the state of the firstuser; the indicator includes one or more of an icon, a color, an image,a textual descriptor, or an audio signal; the biometric data indicatesone or more of heart rate, pulse, perspiration level, blood pressure,galvanic skin response, pupil dilation, or neural oscillation activity;the input data includes audio data of speech of the first user; theoutput data includes a text transcription of at least a portion of theaudio data; and/or transforming the input data to generate the outputdata employs a convolutional neural network including at least one layerthat performs an analysis based on the state of the first user.

Other implementations of any of the above aspects include correspondingsystems, apparatus, and computer programs that are configured to performthe actions of the methods, encoded on computer storage devices. Thepresent disclosure also provides a computer-readable storage mediumcoupled to one or more processors and having instructions stored thereonwhich, when executed by the one or more processors, cause the one ormore processors to perform operations in accordance with implementationsof the methods provided herein. The present disclosure further providesa system for implementing the methods provided herein. The systemincludes one or more processors, and a computer-readable storage mediumcoupled to the one or more processors having instructions stored thereonwhich, when executed by the one or more processors, cause the one ormore processors to perform operations in accordance with implementationsof the methods provided herein.

Implementations of the present disclosure provide one or more of thefollowing technical advantages and improvements over traditionalsystems. Traditional translation methods may focus on the words that arecommunicated and may fail to account for the intent and/or contextunderlying the words. Accordingly, using traditional methods the state,intent, tone, and/or other context may be lost in translation.Implementations improve on traditional methods through the use ofbiometric data to determine a current emotional state of the person whospoke or wrote the input. The emotional state is employed during thetranslation process, as described below, to preserve the tone, intent,and/or context of the person when they spoke or wrote the input data.Accordingly, through the use of biometrically determined emotional stateinformation for a sending user, implementations provide for translationof input data that may be more accurate than translation usingtraditional techniques. Accordingly, implementations make more efficientuse of computing resources such as processing capacity, memory, storage,and/or network bandwidth compared to traditional translation systemsthat may consume computing resources through multiple failed attempts attranslation.

Moreover, traditional methods for communicating speech and/or text databetween users may not convey tone or intent. For example, using atraditional messaging system a sending user may provide input speechdata which is transcribed to text for presentation in a receiving user'smessaging client application. As another example, users may beexchanging text messages using a messaging and/or chat service. Ineither scenario, the sending user's tone, intent, and/or other contextmay be lost in the transcription and/or communication process, such thatthe receiver has no way of knowing when the sender is being earnestand/or literal in their communication, compared to instances when thesender may be sarcastic, ironic, or otherwise attempting to convey somenon-literal nuance with their input data. Implementations also employemotional state information to provide additional context in suchsituations, by presenting output data to the receiving user along withan indication of the sending user's emotional state when they input thedata to be sent. Accordingly, implementations provide an improvementover traditional techniques for natural language translation, text-basedcommunication, and text-based communication with a speech inputinterface, by using a biometrically determined emotional state of thesending user to preserve the intent, tone, nuance, and/or other contextthat may otherwise be lost using traditional techniques. Accordingly,implementations make more efficient use of computing resources such asnetwork bandwidth compared to traditional messaging systems, in whichusers may be required to send one or more follow-up messages to clarifyan initial message that is unclear in its emotional context and/orintent.

It is appreciated that aspects and features in accordance with thepresent disclosure can include any combination of the aspects andfeatures described herein. That is, aspects and features in accordancewith the present disclosure are not limited to the combinations ofaspects and features specifically described herein, but also include anycombination of the aspects and features provided.

The details of one or more implementations of the present disclosure areset forth in the accompanying drawings and the description below. Otherfeatures and advantages of the present disclosure will be apparent fromthe description and drawings, and from the claims.

BRIEF DESCRIPTION OF DRAWINGS

FIG. 1 depicts an example system for data transformation based onbiometric sensor data analysis, according to implementations of thepresent disclosure.

FIG. 2 depicts a flow diagram of an example process for detectingincorrect translation based on biometric sensor data analysis, accordingto implementations of the present disclosure.

FIG. 3 depicts a flow diagram of an example process for translationbased on biometric sensor data analysis, according to implementations ofthe present disclosure.

FIG. 4 depicts a flow diagram of an example process for providing a userstate indicator for transformed data based on biometric sensor dataanalysis, according to implementations of the present disclosure.

FIGS. 5A and 5B depict an example of a user state indicator presented ina user interface with transformed data, according to implementations ofthe present disclosure.

FIG. 6 depicts an example computing system, according to implementationsof the present disclosure.

DETAILED DESCRIPTION

Implementations of the present disclosure are directed to datatransformation such as a natural language (NL) translation that isperformed at least partly based on a current emotional state of the userwho provided the input data, the emotional state determined based onbiometric data for the user. One or more sensors in proximity to a usermay generate biometric data that indicates one or more physiologicalcharacteristics of the user, such as the user's heart rate, pulse, bloodpressure, perspiration level, brain wave activity, pupil dilation,galvanic skin response, and so forth. An emotional state of the user maybe determined based on the biometric data. For example, biometric datamay be used to determine whether the user is happy, sad, angry, calm,agitated, nervous, stressed, relaxed, and/or in some other state. Theuser may provide input data such as audio input (e.g., speech) and/ortext input to be communicated to another user, e.g., during a textmessaging (e.g., chat) session or other scenario. The determinedemotional state of the sending user may be used to transform the inputdata to generate output data that is communicated to the receiving user.

In some implementations, the input data may be translated from one NL toanother NL, and the current emotional state of the sending user may beused to determine particular dictionary entries to use during thetranslation. For example, a particular word in a first language maytranslate to different words in a second language depending on theemotional context in which the input word is uttered. Implementationsmay access different dictionaries and/or different dictionary entries touse in the translation, depending on the emotional state of the senderwhen the word was input. A dictionary entry may provide a mappingbetween a word (or phrase) in the input language and at least one outputword (or phrase) in the output language.

In some implementations, the emotional state of the sending user may beused to infer or otherwise determine that a translation was incorrect,such that another attempt at translation may be needed. For example, a(e.g., bilingual) sending user may see that a word X in an inputlanguage was translated to a word Q in an output language. If thetranslation is not correct, or at least not the output that is ideal ordesired by the sending user, the user's emotional state may becomeangry, frustrated, agitated, and so forth. On detecting that the user isin such a state or has entered such a state during a time when thetranslated output has been presented to the user, implementations mayperform another attempt at translating the word. Any appropriate numberof iterations may be performed until the user's emotional stateindicates a likelihood that they are satisfied with the output of thetranslation.

In some implementations, the input data may be transformed to includeinformation indicating the current emotional state of the sending userwhen they provided the input data. For example, the sending user may beusing a speech-to-text feature of a chat program or messaging clientwhile communicating with the receiving user. Implementations maytranscribe the user's speech input to generate output text, which isthen communicated for presentation to the receiving user. The emotionalstate of the sending user may be monitored, and an indication of thesending user's emotional state may be sent with the transcription. Theoutput text may be presented, along with an indication of the sender'semotional state, to provide the receiving user with some contextregarding the sender's emotional state. For example, the output text maybe presented in a user interface (UI) with an icon, image, color, and/orother indication of the sender's emotional state when they entered theinput data.

Traditional translation methods may focus on the words that arecommunicated and may fail to account for the intent and/or contextunderlying the words. A traditional algorithm may receive input from auser and individually translate the words or phrases of the input usinga dictionary. The dictionaries that are traditionally used in NLtranslation processes may be more or less comprehensive. Although muchresearch has been performed to configure dictionaries for translatingwords and/or phrases from one NL to another, even the most comprehensivedictionary may fail to account for the emotional state, intent, tone,and/or other current context of the person who spoke or wrote the inputdata to be translated. Accordingly, using traditional methods the state,intent, tone, and/or other context may be lost in translation.Implementations improve on traditional methods through the use ofbiometric data to determine a current emotional state of the person whospoke or wrote the input. The emotional state is employed during thetranslation process, as described below, to preserve the tone, intent,and/or context of the person when they spoke or wrote the input data.

Moreover, traditional methods for communicating speech and/or text databetween users may not convey tone or intent. For example, using atraditional messaging system a sending user may provide input speechdata which is transcribed to text for presentation in a receiving user'smessaging client application. As another example, users may beexchanging text messages using a messaging and/or chat service. Ineither scenario, the sending user's tone, intent, and/or other contextmay be lost in the transcription and/or communication process, such thatthe receiver has no way of knowing when the sender is being earnestand/or literal in their communication, compared to instances when thesender may be sarcastic, ironic, or otherwise attempting to convey somenon-literal nuance with their input data. Implementations also employemotional state information to provide additional context in suchsituations, by presenting output data to the receiving user along withan indication of the sending user's emotional state when they input thedata to be sent. Accordingly, implementations provide an improvementover traditional techniques for NL translation, text-basedcommunication, and text-based communication with a speech inputinterface, by using a biometrically determined emotional state of thesending user to preserve the intent, tone, nuance, and/or other contextthat may otherwise be lost using traditional techniques.

FIG. 1 depicts an example system for data transformation based onbiometric sensor data analysis, according to implementations of thepresent disclosure. As shown in the example, two users 102(1) and 102(2)may respectively use user devices 104(1) and 104(2) to communicate withone another. In some instances, the communications may be exchangedduring a communication session in which any number of communications(e.g., messages) are sent and received between the users 102. A sessionmay include any suitable number of communications over a period of time,and there may be a delay between the communications during a session. Insome instances, communications may overlap such that a user 102(1) isreading or hearing a communication from the user 102(2) while the user102(1) is speaking or composing another communication to the user102(2). In some instances, the communications may be audiocommunications, e.g., during a telephone conversation, voice chatsession, video chat session, an exchange of voice mail messages, and soforth. In some instances, the communications may be textualcommunications, e.g., during a chat session using a chat or messagingclient, an exchange of Short Message Service (SMS) and/or MultimediaMessaging Service (MMS) messages, and so forth. Implementations alsosupport other types of communications.

The user devices 104(1) and 104(2) may include any suitable type ofcomputing device, including portable computing device(s) (e.g.,smartphone, tablet computer, wearable computer, etc.) and less portablecomputing device(s) (e.g., laptop computer, desktop computer, etc.). Theuser device 104(1) and/or user device 104(2) may execute an application112 that facilitates communications between the users 102. For example,the application 112 may be a chat client for entering, sending,receiving, and presenting communications that include text data, audiodata, video data, and/or other types of data.

The user 102(1) (e.g., a sending user) may provide input data 106, suchas text input and/or audio input (e.g., speech). The input data 106 maybe received by the application 112(1) executing on the user device104(1). The application 112(1) may also receive sensor data 110generated by various sensor(s) 108 in proximity to the user 102(1). Thesensor(s) 108 may also be described as sensor device(s). The sensor(s)108 proximal to the user 102(1) may include sensor(s) that are inphysical contact with the user 102(1), such as sensor(s) that are wornby the user 102(1) and/or implanted into the user's body. The proximalsensor(s) 108 may also include sensor(s) that are situated externally tothe user 102(1) but in proximity to the user 102(1), e.g., near enoughto collect image data, video data, audio data, thermal data, chemicaldata (e.g., odor information), and/or other data regarding the user102(1).

In some implementations, the sensor data 110 may include sensor data110(1) that is generated by sensor(s) 108(1) that are external to theuser device 104(1) and in proximity to the user 102(1). Such sensor(s)108(1) may be included in a wearable device such as a fitness trackingdevice, and the sensor(s) 108(1) may communicate the sensor data 110(1)to the user device 104(1) over one or more (e.g., wireless) networksusing BlueTooth™, BlueTooth Low Energy™, or any other suitable networkcommunication protocol. The sensor data 110 may include sensor data110(2) that is generated by sensor(s) 108(2) that are incorporated intothe user device 104(1) and in proximity to the user 102(1).

In some implementations, the sensor data 110 (e.g., sensor data 110(1)and/or 110(2)) includes biometric data that describes a physiologicalstate and/or characteristic of the user 102(1). For example, the sensordata 110 may indicate a current value (e.g., when the user composesand/or provides the data to be communicated) of one or more of thefollowing characteristics of the user 102(1): heart rate, pulse, bloodpressure, perspiration level, blood sugar level, body temperature,respiration rate, pupil dilation, galvanic skin response (e.g.,electrodermal activity), and/or brain wave activity (e.g.,neuro-electrical activity). In some implementations, the biometric datamay include data describing voluntary and/or involuntary movements ofthe user 102(1) such as jitters, tremors, eye movements, nervousmovements (e.g., tapping figures), and so forth.

The user device 104(1) may execute one or more sensor data analysismodules 114. The sensor data analysis module(s) 114 may execute assub-module(s) of the application 112(1) (e.g., as shown in FIG. 1). Insome implementations, the sensor data analysis module(s) 114 may executeseparately, e.g., as separate processes, relative to the application112(1). The sensor data analysis module(s) 114 may receive the sensordata 110, e.g., sensor data 110(1) and/or sensor data 110(2), andanalyze the sensor data 110 to determine a current user state 116 of theuser 102(1) based on the sensor data 110. In some implementations, theuser state 116 may be an emotional state of the user 102(1), indicatingwhether the user 102(1) is happy, sad, angry, calm, agitated, nervous,under stress, tired, sleepy, alert, and/or in some other state(s).

In some implementations, the biometric data indicating a physiologicaland/or health status of the user 102(1) may be collected and analyzed todetermine the user's level of stress, anger, happiness, and/or otheruser state(s) 116. For example, an increase or other change in theuser's heart rate, pulse, perspiration, galvanic skin response, brainwave activity, or other biometric data may indicate that the user 102(1)is angry, stressed, and/or otherwise agitated. In some implementations,images of at least a portion of the user's face (e.g., mouth, eyes,etc.) or posture (e.g., shoulders) may be analyzed using moodrecognition analysis techniques to determine the emotional state (e.g.,stress level) of the user 102(1).

In some implementations, at least a portion of the input data 106 mayalso be analyzed to determine the user state 116, in addition to orinstead of analyzing the sensor data 110. For example, audio data of thevoice of the user 102(1) may be analyzed using audio analysis techniquesto detect mood and/or stress indicators in the user's voice and/orlanguage usage. In some implementations, audio data of the voice of theuser 102(1) may be transcribed using speech-to-text (STT) software, andthe generated text may be analyzed using natural language processing(NLP), semantic analysis, keyword recognition, or other methods todetect indications of stress, anger, and/or other emotions in the user'slanguage. In some implementations, text data communicated by the user102(1) may be analyzed to determine a user state 116 of the user 102(1).For example, heightened emotions (e.g., anger) may lead to moremisspellings or grammar errors, use of different vocabulary, use of ahigher proportion of shorter words, different punctuation, and so forth.A change in the emotional state of the user 102(1) may also lead to anincrease or decrease in typing speed, which may be detected and used todetermine the user state 116.

The user device 104(1) may execute one or more transformation modules118 that operate to transform the input data 106 to output data 120. Thetransformation module(s) 118 may execute as sub-module(s) of theapplication 112(1) (e.g., as shown in FIG. 1). In some implementations,the transformation module(s) 118 may execute separately, e.g., asseparate processes, relative to the application 112(1). Thetransformation module(s) 118 may transform the input data 106 togenerate the output data 120, and such transformation may be based atleast partly on the current user state 116 of the user 102(1). In someimplementations, transformation may include translating at least aportion of the input data 106 that is in a first NL to generate theoutput data 120 in a second NL. Such implementations are describedfurther with reference to FIGS. 2 and 3. In some implementations,transformation may include incorporating an emotional indicator into theinput data 106 and/or into a text transcription of the input data 106.Such implementations are described further with reference to FIGS. 4 and5.

The generated output data 120 may be communicated over one or morenetworks to a user device 104(2) of a receiving user 102(2). The outputdata 120 may be presented in a UI of an application 112(2) executing onthe user device 104(2). Although the example of FIG. 1 depicts, for thesake of clarity, a one-way communication between the user device 104(1)and the user device 104(2), implementations are not so limited. In someinstances, the users 102(1) and 102(2) may communicate back-and-forthusing the user devices 104 and the applications 112, e.g., during a textchat or voice chat session. Accordingly, the user device 104(2) and/orapplication 112(2) may include the various components depicted in FIG. 1on the sending side, such as the sensor(s) 108, the sensor data analysismodule(s) 114, and/or the transformation module(s) 118. The user 102(2)may respond to the user 102(1) by providing their own input data, whichmay be transformed to output data based on the user state 116 of theuser 102(2). Such output data may be communicated to the user 102(1)and/or user device 104(1).

In some implementations, the transformation module(s) 118 may employ oneor more machine learning (ML) techniques to generate the output data 120based on the user state 116. Such ML techniques may include, asappropriate, supervised and/or unsupervised ML technique(s). In someimplementations, the transformation module(s) 118 may employconvolutional neural network(s) (CNN(s)) to determine the output data120. A CNN is a type of artificial neural network (ANN), a machinelearning framework that may employ layers of feature extraction followedby a classifier of these features. In some implementations, theclassifier is replaced with a Support Vector Machine (SVM) for improvedclassification in certain applications. In some implementations, the CNNincludes layers for speech and/or text recognition, intent trackingbased on the user state 116, language translation, and/ordictionary-based verification. In this way, implementations may bedescribed as employing a deep learning algorithm that employs multiplelayers of processing, in which each layer is itself a ML algorithm. Insome implementations, the transformation module(s) 118 may employ someother type of ANN, such as an ANN that employs data sampling that ismore regular (e.g., non-stochastic) than the sampling which may be usedin a CNN.

In some implementations, these layers are followed by a Support MatrixMachine (SMM) classifier, which may be employed instead of a SVMclassifier. A SMM classifier may perform analysis based on a matrix ofdimensions (e.g., M by N matrix) instead of a SVM classifier which mayperform analysis based on a vector (e.g., 1 by N vector). Using a SVMclassifier, the process may learn from the analyzed data (e.g., inputdata and/or user state) sequentially, e.g., analyzing one dimension at atime. Using a SMM classifier, the process may learn (e.g.,simultaneously) across multiple dimensions by analyzing the correlationsbetween dimensions. For example, a SMM may be used to correlate therelationship(s) between spoken or written text and an understanding ofthe language intent as indicated by the user state, and suchcorrelation(s) may be correlated with possible translations of the inputdata.

Implementations may employ a SMM instead of a SVM, given that a SVM maybe limited in scope by processing each portion of data individually. ASMM may enable the process to develop a deeper understanding of theadjacent data relative to the analyzed data, and determine relationshipsand/or context between the various dimensions of data being analyzed. Insome implementations, a reduced version of a SMM may be employed. Such aSMM may include the dimensions that have the largest effect on theresult. For example, a customer may call a service representative todiscuss an insurance claim, and the dimensions that are correlated in aSVM may include the customer data and their claim data. Use of a SMM inthis instance accounts for additional correlations between the currentcall and the customer's previous call and/or claim history. Suchcorrelation(s) may help to identify instances of fraud, such asfraudulent insurance claims.

In some implementations, a model of the user 102(1) may be developedover time, the model indicating the typical user state 116 (e.g.,emotional state) of the user 102(1). The model may be developed andrefined based on the sensor data 110 collected that describes the user'sphysiological characteristics. In some implementations, an initialbaselining and/or calibration may be performed to determine an initialmodel for the user 102(1). Alternatively, the model may be created whenthe user 102(1) begins user the user device 104(1) and/or application112(1), and the model may be developed over time as additional sensordata 110 is collected and analyzed regarding the user 102(1). In someimplementations, a current user state 116 of the user 102(1) may berelative to the typical user state indicated by the model of the user102(1), e.g., whether the user is more or less angry than typical, moreor less happy than typical, and so forth.

FIG. 2 depicts a flow diagram of an example process for detectingincorrect translation and/or refining translation based on biometricsensor data analysis, according to implementations of the presentdisclosure. Operations of the process may be performed by one or more ofthe application 112, the sensor data analysis module(s) 114, thetransformation module(s) 118, and/or other software module(s) executingon the user device(s) 104 or elsewhere.

The input data 106 may be received (202). As described above, input data106 may include text data and/or audio data (e.g., speech input)provided by the user 102(1).

The input data 106 may be translated (204) from a first NL to a secondNL. In some implementations, the second NL may be specified by the user102(1). In some implementations, the second NL may be determined basedon language preferences, location, and/or other characteristic(s) of theuser 102(2) who is to receive the output data 120.

The sensor data 110 may be received (206). As described above, thesensor data 110 may include biometric data describing variousphysiological characteristic(s) of the user 102(1), and may be generatedby one or more sensors 108 in proximity to the user 102(1).

The user state 116 of the user 102(1) may be determined (208) based onan analysis of the sensor data 110. As described above, the user state116 may be an emotional state of the user 102(1). In someimplementations, the sensor data 110 may be analyzed in real time as itis collected. Accordingly, the user state 116 may be a current userstate of the user 102(1), current with respect to a time period duringwhich the sensor data 110 is collected and analyzed. Moreover, thesensor data 110 may be collected and analyzed in real time with respectto the user providing the input data 106. The sensor data 110 may becollected and/or analyzed during a time period that includes, and/or isproximal in time to, a time when the user entered the input data 106 tobe communicated. Accordingly, the user state 116 may be a current userstate 116 of the user 102(1) when the user 102(1) is entering the inputdata 106, such as the state of the user when they entered a text messageto be sent to another user, or spoke a phrase to be translated and/ortranscribed and communicated to another user. Real time operation(s) mayinclude the automatic performing of one or more operations withoutrequiring human input and without any intentional delay with respect toa triggering event, taking into account the processing limitations ofthe computing system(s) performing the operations and the time needed toperform the operations. Accordingly, the analysis of the sensor data 110may be performed in real time with respect to the collection of thesensor data 110, and the collection and/or analysis of the sensor data110 may be performed in real time with respect to the user providing theinput data 106 as speech and/or text input.

The output of the translation may be presented to the user 102(1). Adetermination may be made (210) whether the user's current stateindicates that the translation may have been incorrect. For example, ifthe user's emotional state becomes angry, agitated, frustrated, and/orstressed when the translation is presented, an inference may be madethat the translation is incorrect or in some way inappropriate. In suchinstances, the process may return to 204 and another attempt may be madeto translate at least a portion of the input data 106 (e.g., usingdifferent dictionary entries).

If the user's emotional state stays the same when the translation ispresented, or exhibits a positive change (e.g., the user becomes happy,calm, etc.), an inference may be made that the translation is correct orotherwise appropriate. The translated output data 120 may becommunicated (212) for presentation to the user 102(2) on the userdevice 104(2).

In this way, implementations may employ the user state 116 of the user102(1) as feedback to determine whether the translation was correct.Such feedback may also be employed to train, refine, or otherwise refinea ML-based classifier used to perform the translation of the input data106 as described above. In some implementations, the user state 116 ofthe receiving user 102(2) may be monitored based on collected sensordata 110, and the receiving user's emotional state may be employed torefine the translation. For example, if the receiving user's emotionalstate is confused and/or agitated on receiving the translated outputdata 120, a determination may be made that the translation was incorrectand another attempt at translation may be performed on the sendinguser's user device 104(1).

FIG. 3 depicts a flow diagram of an example process for translationbased on biometric sensor data analysis, according to implementations ofthe present disclosure. Operations of the process may be performed byone or more of the application 112, the sensor data analysis module(s)114, the transformation module(s) 118, and/or other software module(s)executing on the user device(s) 104 or elsewhere.

The input data 106 may be received (302). As described above, input data106 may include text data and/or audio data (e.g., speech input)provided by the user 102(1).

The sensor data 110 may be received (304). As described above, thesensor data 110 may include biometric data describing variousphysiological characteristic(s) of the user 102(1), and may be generatedby one or more sensors 108 in proximity to the user 102(1).

The user state 116 of the user 102(1) may be determined (306) based onan analysis of the sensor data 110. As described above, the user state116 may be an emotional state of the user 102(1).

A determination may be made (308) whether there is an availabledictionary that corresponds to the current state of the user 102(1),and/or whether there are one or more dictionary entries that areavailable and that correspond to the current state of the user 102(1).If so, the available dictionary and/or entries may be used (310) fortranslation. If not, a default dictionary and/or entries may be used(314).

The input data 106 may be translated (312) from a first NL to a secondNL using the identified dictionary and/or dictionary entries. In someimplementations, the second NL may be specified by the user 102(1). Insome implementations, the second NL may be determined based on languagepreferences, location, and/or other characteristic(s) of the user 102(2)who is to receive the output data 120.

The translated output data 120 may be communicated (314) forpresentation to the user 102(2) on the user device 104(2).

In some implementations, the process may have access to variousdictionaries that translate from NL X to NL Y, and each dictionary mayinclude entries that are suitable to translate based on a particulardetected emotional state. In some implementations, the process may haveaccess to a dictionary for translating from NL X to NL Y, and thedictionary may include a first set of entries suitable for translationduring a first emotional state, and a second set of entries suitable fortranslation during a second emotional state. For example, a word orphrase in NL X may be appropriately translated to a particular word orphrase in NL Y if the speaker (or writer) is in a happy mood, and thesame word or phrase in NL X may be appropriately translated to adifferent word or phrase in NL Y if the speaker (or write) is in anangry mood. In this way, implementations may employ the currentemotional state of the sending user 102(1) to generate a more accuratetranslation of at least a portion of the user's input data 106 thanwould be otherwise generated using a default (e.g., emotion-agnostic)dictionary and/or dictionary entries.

In some implementations, a dictionary may be used which is specificallycreated and adapted for use in modifying the communications of theparticular user (e.g., speaker or writer). In some implementations, thedictionary may be specific to a particular class or category of usersthat include the user, such as a demographic group (e.g., agerange-based group) or location-based group (e.g., group of users in aparticular region, country, city, etc.). The user-specific dictionarymay provide a baseline for the particular user, and may be modified overtime based on detected changes in the user's vocabulary, grammar,syntax, or other changes in written and/or spoken language usage. Insome implementations, a dictionary may be used which is specific to boththe particular user and the detected emotional state for the user. Forexample, a particular user may be associated with multiple dictionariesthat correspond to different emotional states (e.g., angry, happy, sad,etc.). The dictionary that is used for translation and/or some othercommunication modification may be selected for the particular user andtheir current emotional state.

FIG. 4 depicts a flow diagram of an example process for providing a userstate indicator for transformed data based on biometric sensor dataanalysis, according to implementations of the present disclosure.Operations of the process may be performed by one or more of theapplication 112, the sensor data analysis module(s) 114, thetransformation module(s) 118, and/or other software module(s) executingon the user device(s) 104 or elsewhere.

The input data 106 may be received (402). As described above, input data106 may include text data and/or audio data (e.g., speech input)provided by the user 102(1).

In some instances, at least a portion of the input data 106 may betransformed (404). As described above, transformation may includetranslating the input data 106 from a first NL to a second NL. In someinstances, transformation may include transcribing audio input data 106to generate text data, e.g., through use of a speech recognition processand/or STT module.

The sensor data 110 may be received (406). As described above, thesensor data 110 may include biometric data describing variousphysiological characteristic(s) of the user 102(1), and may be generatedby one or more sensors 108 in proximity to the user 102(1).

The user state 116 of the user 102(1) may be determined (408) based onan analysis of the sensor data 110. As described above, the user state116 may be an emotional state of the user 102(1).

The input data 106 may be further transformed to incorporate (410), intothe output data 120, an emotional indicator that corresponds to the userstate 116 of the user 102(1).

The output data 120, including the emotional indicator, may becommunicated (314) for presentation to the user 102(2) on the userdevice 104(2). The output data 120 may be presented along with theemotional indicator, to indicate to the user 102(2) the emotional stateof the sending user 102(1) when the input data 106 was provided. In thisway, implementations may send text data while conveying the emotionalcontext and/or intent of the sending user.

FIGS. 5A and 5B depict an example of a user state (e.g., emotional)indicator presented in a UI 502 with transformed data, according toimplementations of the present disclosure. In some implementations, theUI 502 may be an interface of the application 112(2), such as a chatclient, messaging client, and so forth.

In the example of FIG. 5A, the sending user 102(1) has spoken or typedthe input data “I'm so angry right now,” which is communicated as theoutput data 120 for presentation to the user 102(2) in the UI 502. Inthis example, the sensor data 110 is analyzed to determine that the user102(1) is angry, so that the user statement “I'm so angry right now” ismade when the user 102(1) is actually angry. The output data 120 “I'm soangry right now” is presented, through the UI 502, with an emotionalindicator 504 indicating that the sending user's current state is anger.

In the example of FIG. 5B, the sending user 102(1) has spoken or typedthe same input data “I'm so angry right now,” which is communicated asthe output data 120 for presentation to the user 102(2) in the UI 502.In this example, the sensor data 110 is analyzed to determine that theuser 102(1) is currently happy, amused, or in some other way not angry.In this instance, the user statement “I'm so angry right now” may havebeen made by the user 102(1) ironically, sarcastically, or in some otherway such that the literal meaning of the input data is not what thesending user was actually feeling at the time. The output data 120 “I'mso angry right now” is presented, through the UI 502, with an emotionalindicator 504 indicating that the sending user's current state is happy,amused, or otherwise not angry.

Although FIGS. 5A and 5B depict the emotional indicator 504 as a faceemoticon, implementations are not limited to this example. The emotionalindicator 504 may include one or more of an emoticon, an icon, an emoji,a symbol, a graphic, a color, an image, a textual descriptor, and/or anyother suitable visual indicator. In some implementations, the emotionalindicator 504 may include audio data (e.g., an audio signal), such asone or more sounds or portions of music that indicate an emotionalstate. The emotional indicator 504 may include a visual indicator, anaudio indicator, and/or any suitable combination of visual and/or audioindicator(s).

In some implementations, the emotional indicator 504 may be incorporatedinto the output data 120 and communicated to the user device 104(2) fromthe user device 104(1). In some implementations, the output data 120 maybe transformed to include metadata that indicates an emotional state ofthe sending user 102(1). The metadata may be interpreted by theapplication 112(1) and/or UI 502 and used to determine an appropriateemotional indicator 504 to be presented in the UI 504 with the outputdata 120.

FIG. 6 depicts an example computing system, according to implementationsof the present disclosure. The system 600 may be used for any of theoperations described with respect to the various implementationsdiscussed herein. For example, the system 600 may be included, at leastin part, in one or more of the user device(s) 104 and/or other computingdevice(s) or system(s) described herein. The system 600 may include oneor more processors 610, a memory 620, one or more storage devices 630,and one or more input/output (I/O) devices 650 controllable through oneor more I/O interfaces 640. The various components 610, 620, 630, 640,or 650 may be interconnected through at least one system bus 660, whichmay enable the transfer of data between the various modules andcomponents of the system 600.

The processor(s) 610 may be configured to process instructions forexecution within the system 600. The processor(s) 610 may includesingle-threaded processor(s), multi-threaded processor(s), or both. Theprocessor(s) 610 may be configured to process instructions stored in thememory 620 or on the storage device(s) 630. The processor(s) 610 mayinclude hardware-based processor(s) each including one or more cores.The processor(s) 610 may include general purpose processor(s), specialpurpose processor(s), or both.

The memory 620 may store information within the system 600. In someimplementations, the memory 620 includes one or more computer-readablemedia. The memory 620 may include any number of volatile memory units,any number of non-volatile memory units, or both volatile andnon-volatile memory units. The memory 620 may include read-only memory,random access memory, or both. In some examples, the memory 620 may beemployed as active or physical memory by one or more executing softwaremodules.

The storage device(s) 630 may be configured to provide (e.g.,persistent) mass storage for the system 600. In some implementations,the storage device(s) 630 may include one or more computer-readablemedia. For example, the storage device(s) 630 may include a floppy diskdevice, a hard disk device, an optical disk device, or a tape device.The storage device(s) 630 may include read-only memory, random accessmemory, or both. The storage device(s) 630 may include one or more of aninternal hard drive, an external hard drive, or a removable drive.

One or both of the memory 620 or the storage device(s) 630 may includeone or more computer-readable storage media (CRSM). The CRSM may includeone or more of an electronic storage medium, a magnetic storage medium,an optical storage medium, a magneto-optical storage medium, a quantumstorage medium, a mechanical computer storage medium, and so forth. TheCRSM may provide storage of computer-readable instructions describingdata structures, processes, applications, programs, other modules, orother data for the operation of the system 600. In some implementations,the CRSM may include a data store that provides storage ofcomputer-readable instructions or other information in a non-transitoryformat. The CRSM may be incorporated into the system 600 or may beexternal with respect to the system 600. The CRSM may include read-onlymemory, random access memory, or both. One or more CRSM suitable fortangibly embodying computer program instructions and data may includeany type of non-volatile memory, including but not limited to:semiconductor memory devices, such as EPROM, EEPROM, and flash memorydevices; magnetic disks such as internal hard disks and removable disks;magneto-optical disks; and CD-ROM and DVD-ROM disks. In some examples,the processor(s) 610 and the memory 620 may be supplemented by, orincorporated into, one or more application-specific integrated circuits(ASICs).

The system 600 may include one or more I/O devices 650. The I/Odevice(s) 650 may include one or more input devices such as a keyboard,a mouse, a pen, a game controller, a touch input device, an audio inputdevice (e.g., a microphone), a gestural input device, a haptic inputdevice, an image or video capture device (e.g., a camera), or otherdevices. In some examples, the I/O device(s) 650 may also include one ormore output devices such as a display, LED(s), an audio output device(e.g., a speaker), a printer, a haptic output device, and so forth. TheI/O device(s) 650 may be physically incorporated in one or morecomputing devices of the system 600, or may be external with respect toone or more computing devices of the system 600.

The system 600 may include one or more I/O interfaces 640 to enablecomponents or modules of the system 600 to control, interface with, orotherwise communicate with the I/O device(s) 650. The I/O interface(s)640 may enable information to be transferred in or out of the system600, or between components of the system 600, through serialcommunication, parallel communication, or other types of communication.For example, the I/O interface(s) 640 may comply with a version of theRS-232 standard for serial ports, or with a version of the IEEE 1284standard for parallel ports. As another example, the I/O interface(s)640 may be configured to provide a connection over Universal Serial Bus(USB) or Ethernet. In some examples, the I/O interface(s) 640 may beconfigured to provide a serial connection that is compliant with aversion of the IEEE 1394 standard.

The I/O interface(s) 640 may also include one or more network interfacesthat enable communications between computing devices in the system 600,or between the system 600 and other network-connected computing systems.The network interface(s) may include one or more network interfacecontrollers (NICs) or other types of transceiver devices configured tosend and receive communications over one or more networks using anynetwork protocol.

Computing devices of the system 600 may communicate with one another, orwith other computing devices, using one or more networks. Such networksmay include public networks such as the internet, private networks suchas an institutional or personal intranet, or any combination of privateand public networks. The networks may include any type of wired orwireless network, including but not limited to local area networks(LANs), wide area networks (WANs), wireless WANs (WWANs), wireless LANs(WLANs), mobile communications networks (e.g., 3G, 4G, Edge, etc.), andso forth. In some implementations, the communications between computingdevices may be encrypted or otherwise secured. For example,communications may employ one or more public or private cryptographickeys, ciphers, digital certificates, or other credentials supported by asecurity protocol, such as any version of the Secure Sockets Layer (SSL)or the Transport Layer Security (TLS) protocol.

The system 600 may include any number of computing devices of any type.The computing device(s) may include, but are not limited to: a personalcomputer, a smartphone, a tablet computer, a wearable computer, animplanted computer, a mobile gaming device, an electronic book reader,an automotive computer, a desktop computer, a laptop computer, anotebook computer, a game console, a home entertainment device, anetwork computer, a server computer, a mainframe computer, a distributedcomputing device (e.g., a cloud computing device), a microcomputer, asystem on a chip (SoC), a system in a package (SiP), and so forth.Although examples herein may describe computing device(s) as physicaldevice(s), implementations are not so limited. In some examples, acomputing device may include one or more of a virtual computingenvironment, a hypervisor, an emulation, or a virtual machine executingon one or more physical computing devices. In some examples, two or morecomputing devices may include a cluster, cloud, farm, or other groupingof multiple devices that coordinate operations to provide loadbalancing, failover support, parallel processing capabilities, sharedstorage resources, shared networking capabilities, or other aspects.

Implementations and all of the functional operations described in thisspecification may be realized in digital electronic circuitry, or incomputer software, firmware, or hardware, including the structuresdisclosed in this specification and their structural equivalents, or incombinations of one or more of them. Implementations may be realized asone or more computer program products, i.e., one or more modules ofcomputer program instructions encoded on a computer readable medium forexecution by, or to control the operation of, data processing apparatus.The computer readable medium may be a machine-readable storage device, amachine-readable storage substrate, a memory device, a composition ofmatter effecting a machine-readable propagated signal, or a combinationof one or more of them. The term “computing system” encompasses allapparatus, devices, and machines for processing data, including by wayof example a programmable processor, a computer, or multiple processorsor computers. The apparatus may include, in addition to hardware, codethat creates an execution environment for the computer program inquestion, e.g., code that constitutes processor firmware, a protocolstack, a database management system, an operating system, or acombination of one or more of them. A propagated signal is anartificially generated signal, e.g., a machine-generated electrical,optical, or electromagnetic signal that is generated to encodeinformation for transmission to suitable receiver apparatus.

A computer program (also known as a program, software, softwareapplication, script, or code) may be written in any appropriate form ofprogramming language, including compiled or interpreted languages, andit may be deployed in any appropriate form, including as a standaloneprogram or as a module, component, subroutine, or other unit suitablefor use in a computing environment. A computer program does notnecessarily correspond to a file in a file system. A program may bestored in a portion of a file that holds other programs or data (e.g.,one or more scripts stored in a markup language document), in a singlefile dedicated to the program in question, or in multiple coordinatedfiles (e.g., files that store one or more modules, sub programs, orportions of code). A computer program may be deployed to be executed onone computer or on multiple computers that are located at one site ordistributed across multiple sites and interconnected by a communicationnetwork.

The processes and logic flows described in this specification may beperformed by one or more programmable processors executing one or morecomputer programs to perform functions by operating on input data andgenerating output. The processes and logic flows may also be performedby, and apparatus may also be implemented as, special purpose logiccircuitry, e.g., an FPGA (field programmable gate array) or an ASIC(application specific integrated circuit).

Processors suitable for the execution of a computer program include, byway of example, both general and special purpose microprocessors, andany one or more processors of any appropriate kind of digital computer.Generally, a processor may receive instructions and data from a readonly memory or a random access memory or both. Elements of a computercan include a processor for performing instructions and one or morememory devices for storing instructions and data. Generally, a computermay also include, or be operatively coupled to receive data from ortransfer data to, or both, one or more mass storage devices for storingdata, e.g., magnetic, magneto optical disks, or optical disks. However,a computer need not have such devices. Moreover, a computer may beembedded in another device, e.g., a mobile telephone, a personal digitalassistant (PDA), a mobile audio player, a Global Positioning System(GPS) receiver, to name just a few. Computer readable media suitable forstoring computer program instructions and data include all forms ofnon-volatile memory, media and memory devices, including by way ofexample semiconductor memory devices, e.g., EPROM, EEPROM, and flashmemory devices; magnetic disks, e.g., internal hard disks or removabledisks; magneto optical disks; and CD ROM and DVD-ROM disks. Theprocessor and the memory may be supplemented by, or incorporated in,special purpose logic circuitry.

To provide for interaction with a user, implementations may be realizedon a computer having a display device, e.g., a CRT (cathode ray tube) orLCD (liquid crystal display) monitor, for displaying information to theuser and a keyboard and a pointing device, e.g., a mouse or a trackball,by which the user may provide input to the computer. Other kinds ofdevices may be used to provide for interaction with a user as well; forexample, feedback provided to the user may be any appropriate form ofsensory feedback, e.g., visual feedback, auditory feedback, or tactilefeedback; and input from the user may be received in any appropriateform, including acoustic, speech, or tactile input.

Implementations may be realized in a computing system that includes aback end component, e.g., as a data server, or that includes amiddleware component, e.g., an application server, or that includes afront end component, e.g., a client computer having a graphical UI or aweb browser through which a user may interact with an implementation, orany appropriate combination of one or more such back end, middleware, orfront end components. The components of the system may be interconnectedby any appropriate form or medium of digital data communication, e.g., acommunication network. Examples of communication networks include alocal area network (“LAN”) and a wide area network (“WAN”), e.g., theInternet.

The computing system may include clients and servers. A client andserver are generally remote from each other and typically interactthrough a communication network. The relationship of client and serverarises by virtue of computer programs running on the respectivecomputers and having a client-server relationship to each other.

While this specification contains many specifics, these should not beconstrued as limitations on the scope of the disclosure or of what maybe claimed, but rather as descriptions of features specific toparticular implementations. Certain features that are described in thisspecification in the context of separate implementations may also beimplemented in combination in a single implementation. Conversely,various features that are described in the context of a singleimplementation may also be implemented in multiple implementationsseparately or in any suitable sub-combination. Moreover, althoughfeatures may be described above as acting in certain combinations andeven initially claimed as such, one or more features from a claimedcombination may in some examples be excised from the combination, andthe claimed combination may be directed to a sub-combination orvariation of a sub-combination.

Similarly, while operations are depicted in the drawings in a particularorder, this should not be understood as requiring that such operationsbe performed in the particular order shown or in sequential order, orthat all illustrated operations be performed, to achieve desirableresults. In certain circumstances, multitasking and parallel processingmay be advantageous. Moreover, the separation of various systemcomponents in the implementations described above should not beunderstood as requiring such separation in all implementations, and itshould be understood that the described program components and systemsmay generally be integrated together in a single software product orpackaged into multiple software products.

A number of implementations have been described. Nevertheless, it willbe understood that various modifications may be made without departingfrom the spirit and scope of the disclosure. For example, various formsof the flows shown above may be used, with steps re-ordered, added, orremoved. Accordingly, other implementations are within the scope of thefollowing claims.

What is claimed is:
 1. A computer-implemented method performed by atleast one processor, the method comprising: receiving, by the at leastone processor, textual data that is typed into a chat client, by a firstuser during a time period, for communication to a second user, whereinthe textual data comprises first natural language data; determining, bythe at least one processor, a state of the first user during the timeperiod when the textual data is typed into the chat client, the statedetermined based on biometric data that is generated by at least onesensor device measuring at least one physiological characteristic of thefirst user during the time period, wherein the at least one sensordevice comprises a wearable device separate from the at least oneprocessor; transforming, by the at least one processor, the textual datato generate transformed textual data and an emotional indicator based atleast partly on the state, wherein the transformed textual datacomprises second natural language data, the emotional indicatorcomprises an icon, a color, an image, a textual descriptor, an audiosignal, or any combination thereof, corresponding to the state of thefirst user, and transforming the textual data to generate thetransformed textual comprises: determining whether one or moredictionary entries correspond to the state of the first user;transforming the textual data into the transformed textual data usingthe one or more dictionary entries in response to the one or moredictionary entries corresponding to the state of the first user; andtransforming the textual data into the transformed textual data usingone or more default dictionary entries in response to the one or moredictionary entries not corresponding to the state of the first user,wherein the one or more default dictionary entries are emotion-agnostic;and communicating, by the at least one processor, the transformedtextual data and the emotional indicator to be presented to the seconduser through an electronic display of a computing device.
 2. The methodof claim 1, wherein transforming the textual data includes translatingthe textual data from the first natural language data to generate thetransformed textual data in the second natural language data.
 3. Themethod of claim 2, wherein the translating includes: performing a firsttranslation of at least a portion of the textual data from the firstnatural language data to generate at least a portion of the transformedtextual data in the second natural language data; determining, based onan updated state of the first user, that the first translation isincorrect; and performing, based on the updated state of the first user,a second translation of at least the portion of the textual data fromthe first natural language data to generate at least the portion of thetransformed textual data in the second natural language data.
 4. Themethod of claim 3, comprising presenting the first translation forviewing by the first user during an additional time period.
 5. Themethod of claim 4, comprising determining the updated state of the firstuser during the additional time period based on additional biometricdata that is generated by the at least one sensor device measuring theat least one physiological characteristic of the first user during theadditional time period.
 6. The method of claim 1, wherein thetransformed textual data is generated to include metadata thatcorresponds to the emotional indicator, the state of the first user, orboth.
 7. The method of claim 1, wherein the biometric data indicates aheart rate, a pulse, a perspiration level, a blood pressure, a galvanicskin response, a pupil dilation, a neural oscillation activity, or anycombination thereof.
 8. The method of claim 1, wherein transforming thetextual data to generate the transformed textual data employs aconvolutional neural network including at least one layer that performsan analysis based on the state of the first user.
 9. The method of claim1, wherein the emotional indicator comprises an emoticon, an emoji, orboth.
 10. The method of claim 9, wherein communicating the emotionalindicator to be presented to the second user through a computing devicecomprises presenting the emoticon, the emoji, or both, in another chatclient for viewing by the second user.
 11. The method of claim 1,wherein the one or more dictionary entries correspond to a category ofusers including the first user, and the category of users comprises ademographic-based category, a location-based category, or both.
 12. Themethod of claim 1, comprising determining, by the at least oneprocessor, the state of the first user based on an increase or adecrease in typing speed of the first user over the time period.
 13. Asystem, comprising: at least one processor; and a memory communicativelycoupled to the at least one processor, the memory storing instructionswhich, when executed by the at least one processor, cause the at leastone processor to perform operations comprising: receiving textual datathat is typed into a chat client, by a first user during a time period,for communication to a second user, wherein the textual data comprisesfirst natural language data; determining a state of the first userduring the time period when the textual data is typed into the chatclient, the state determined based on biometric data that is generatedby at least one sensor device measuring at least one physiologicalcharacteristic of the first user during the time period, wherein the atleast one sensor device comprises a wearable device separate from the atleast one processor; transforming the textual data to generatetransformed textual data and an emotional indicator based at leastpartly on the state, wherein the transformed textual data comprisessecond natural language data, the emotional indicator comprises an icon,a color, an image, a textual descriptor, an audio signal, or anycombination thereof, corresponding to the state of the first user, andtransforming the textual data to generate the transformed textualcomprises: determining whether one or more dictionary entries correspondto the state of the first user; transforming the textual data into thetransformed textual data using the one or more dictionary entries inresponse to the one or more dictionary entries corresponding to thestate of the first user; and transforming the textual data into thetransformed textual data using one or more default dictionary entries inresponse to the one or more dictionary entries not corresponding to thestate of the first user, wherein the one or more default dictionaryentries are emotion-agnostic; and communicating the transformed textualdata and the emotional indicator to be presented to the second userthrough an electronic display of a computing device.
 14. The system ofclaim 13, wherein transforming the textual data includes translating thetextual data from the first natural language data to generate thetransformed textual data in the second natural language data.
 15. Thesystem of claim 14, wherein the translating includes: performing a firsttranslation of at least a portion of the textual data from the firstnatural language data to generate at least a portion of the transformedtextual data in the second natural language data; determining, based onan updated state of the first user, that the first translation isincorrect; and performing, based on the updated state of the first user,a second translation of at least the portion of the textual data fromthe first natural language data to generate at least the portion of thetransformed textual data in the second natural language data.
 16. Thesystem of claim 13, wherein the transformed textual data is generated toinclude metadata that corresponds to the emotional indicator, the stateof the first user, or both.
 17. The system of claim 13, wherein theemotional indicator comprises an emoticon, an emoji, or both.
 18. One ormore computer-readable media storing instructions which, when executedby at least one processor, cause the at least one processor to performoperations comprising: receiving textual data that is typed into a chatclient, by a first user during a time period, for communication to asecond user, wherein the textual data comprises first natural languagedata; determining a state of the first user during the time period whenthe textual data is typed into the chat client, the state determinedbased on biometric data that is generated by at least one sensor devicemeasuring at least one physiological characteristic of the first userduring the time period, wherein the at least one sensor device comprisesa wearable device separate from the at least one processor; transformingthe textual data to generate transformed textual data and an emotionalindicator based at least partly on the state, wherein the transformedtextual data comprises second natural language data, the emotionalindicator comprises an icon, a color, an image, a textual descriptor, anaudio signal, or any combination thereof, corresponding to the state ofthe first user, and transforming the textual data to generate thetransformed textual comprises: determining whether one or moredictionary entries correspond to the state of the first user;transforming the textual data into the transformed textual data usingthe one or more dictionary entries in response to the one or moredictionary entries corresponding to the state of the first user; andtransforming the textual data into the transformed textual data usingone or more default dictionary entries in response to the one or moredictionary entries not corresponding to the state of the first user,wherein the one or more default dictionary entries are emotion-agnostic;and communicating the transformed textual data and the emotionalindicator to be presented to the second user through an electronicdisplay of a computing device.
 19. The one or more computer-readablemedia of claim 18, wherein the emotional indicator comprises anemoticon, an emoji, or both.